

<!DOCTYPE html>
<html class="writer-html5" lang="en" >
<head>
  <meta charset="utf-8" />
  <meta name="generator" content="Docutils 0.19: https://docutils.sourceforge.io/" />

  <meta name="viewport" content="width=device-width, initial-scale=1.0" />
  
  <title>Ceph 文件系统客户端的驱逐 &mdash; Ceph Documentation</title>
  

  
  <link rel="stylesheet" href="../../_static/ceph.css" type="text/css" />
  <link rel="stylesheet" href="../../_static/pygments.css" type="text/css" />
  <link rel="stylesheet" href="../../_static/pygments.css" type="text/css" />
  <link rel="stylesheet" href="../../_static/ceph.css" type="text/css" />
  <link rel="stylesheet" href="../../_static/graphviz.css" type="text/css" />
  <link rel="stylesheet" href="../../_static/css/custom.css" type="text/css" />

  
  

  
  

  

  
  <!--[if lt IE 9]>
    <script src="../../_static/js/html5shiv.min.js"></script>
  <![endif]-->
  
    
      <script type="text/javascript" id="documentation_options" data-url_root="../../" src="../../_static/documentation_options.js"></script>
        <script src="../../_static/jquery.js"></script>
        <script src="../../_static/_sphinx_javascript_frameworks_compat.js"></script>
        <script data-url_root="../../" id="documentation_options" src="../../_static/documentation_options.js"></script>
        <script src="../../_static/doctools.js"></script>
        <script src="../../_static/sphinx_highlight.js"></script>
    
    <script type="text/javascript" src="../../_static/js/theme.js"></script>

    
    <link rel="index" title="Index" href="../../genindex/" />
    <link rel="search" title="Search" href="../../search/" />
    <link rel="next" title="Ceph 文件系统洗刷" href="../scrub/" />
    <link rel="prev" title="CephFS 快照" href="../snapshots/" /> 
</head>

<body class="wy-body-for-nav">

   
  <header class="top-bar">
    <div role="navigation" aria-label="Page navigation">
  <ul class="wy-breadcrumbs">
      <li><a href="../../" class="icon icon-home" aria-label="Home"></a></li>
          <li class="breadcrumb-item"><a href="../">Ceph 文件系统</a></li>
      <li class="breadcrumb-item active">Ceph 文件系统客户端的驱逐</li>
      <li class="wy-breadcrumbs-aside">
            <a href="../../_sources/cephfs/eviction.rst.txt" rel="nofollow"> View page source</a>
      </li>
  </ul>
  <hr/>
</div>
  </header>
  <div class="wy-grid-for-nav">
    
    <nav data-toggle="wy-nav-shift" class="wy-nav-side">
      <div class="wy-side-scroll">
        <div class="wy-side-nav-search"  style="background: #eee" >
          

          
            <a href="../../" class="icon icon-home"> Ceph
          

          
          </a>

          

          
<div role="search">
  <form id="rtd-search-form" class="wy-form" action="../../search/" method="get">
    <input type="text" name="q" placeholder="Search docs" aria-label="Search docs" />
    <input type="hidden" name="check_keywords" value="yes" />
    <input type="hidden" name="area" value="default" />
  </form>
</div>

          
        </div>

        
        <div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="main navigation">
          
            
            
              
            
            
              <ul class="current">
<li class="toctree-l1"><a class="reference internal" href="../../start/">Ceph 简介</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../install/">安装 Ceph</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../cephadm/">Cephadm</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../rados/">Ceph 存储集群</a></li>
<li class="toctree-l1 current"><a class="reference internal" href="../">Ceph 文件系统</a><ul class="current">
<li class="toctree-l2"><a class="reference internal" href="../#cephfs">CephFS 入门</a></li>
<li class="toctree-l2"><a class="reference internal" href="../#id4">管理</a></li>
<li class="toctree-l2"><a class="reference internal" href="../#id5">挂载 CephFS</a></li>
<li class="toctree-l2"><a class="reference internal" href="../#id6">CephFS 内幕</a></li>
<li class="toctree-l2 current"><a class="reference internal" href="../#id7">故障排除和灾难恢复</a><ul class="current">
<li class="toctree-l3 current"><a class="current reference internal" href="#"> 驱逐客户端</a><ul>
<li class="toctree-l4"><a class="reference internal" href="#id1">自动驱逐客户端</a></li>
<li class="toctree-l4"><a class="reference internal" href="#id2">手动驱逐客户端</a></li>
<li class="toctree-l4"><a class="reference internal" href="#id3">高级话题：从黑名单踢出客户端</a></li>
<li class="toctree-l4"><a class="reference internal" href="#id4">高级话题：配置黑名单功能</a></li>
<li class="toctree-l4"><a class="reference internal" href="#osd">背景： 黑名单和 OSD 元图屏蔽</a></li>
</ul>
</li>
<li class="toctree-l3"><a class="reference internal" href="../scrub/"> 洗刷</a></li>
<li class="toctree-l3"><a class="reference internal" href="../full/"> 文件系统占满的处理</a></li>
<li class="toctree-l3"><a class="reference internal" href="../disaster-recovery-experts/"> 元数据修复</a></li>
<li class="toctree-l3"><a class="reference internal" href="../troubleshooting/"> 故障排除</a></li>
<li class="toctree-l3"><a class="reference internal" href="../disaster-recovery/"> 灾难恢复</a></li>
<li class="toctree-l3"><a class="reference internal" href="../cephfs-journal-tool/"> cephfs-journal-tool</a></li>
<li class="toctree-l3"><a class="reference internal" href="../recover-fs-after-mon-store-loss/"> 监视器存储丢失后恢复文件系统</a></li>
<li class="toctree-l3"><a class="reference internal" href="../#id8">开发者指南</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="../#id9">更多细节</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../../rbd/">Ceph 块设备</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../radosgw/">Ceph 对象网关</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../mgr/">Ceph 管理器守护进程</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../mgr/dashboard/">Ceph 仪表盘</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../monitoring/">监控概览</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../api/">API 文档</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../architecture/">体系结构</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../dev/developer_guide/">开发者指南</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../dev/internals/">Ceph 内幕</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../governance/">项目管理</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../foundation/">Ceph 基金会</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../ceph-volume/">ceph-volume</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../releases/general/">Ceph 版本（总目录）</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../releases/">Ceph 版本（索引）</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../security/">Security</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../hardware-monitoring/">硬件监控</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../glossary/">Ceph 术语</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../jaegertracing/">Tracing</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../translation_cn/">中文版翻译资源</a></li>
</ul>

            
          
        </div>
        
      </div>
    </nav>

    <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap">

      
      <nav class="wy-nav-top" aria-label="top navigation">
        
          <i data-toggle="wy-nav-top" class="fa fa-bars"></i>
          <a href="../../">Ceph</a>
        
      </nav>


      <div class="wy-nav-content">
        
        <div class="rst-content">
        
          <div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
           <div itemprop="articleBody">
            
<div id="dev-warning" class="admonition note">
  <p class="first admonition-title">Notice</p>
  <p class="last">This document is for a development version of Ceph.</p>
</div>
  <div id="docubetter" align="right" style="padding: 5px; font-weight: bold;">
    <a href="https://pad.ceph.com/p/Report_Documentation_Bugs">Report a Documentation Bug</a>
  </div>

  
  <section id="ceph">
<h1>Ceph 文件系统客户端的驱逐<a class="headerlink" href="#ceph" title="Permalink to this heading"></a></h1>
<p>当某个文件系统客户端不响应或者有其它异常行为时，有必要强制切断它到文件系统的访问，这个过程就叫做<em>驱逐</em>。</p>
<p>驱逐 CephFS 客户端，就是防止它再与 MDS 和 OSD 守护进程通讯。如果客户端已经对文件系统的缓冲 IO 做了些操作，这些未刷回的数据会丢失。</p>
<p>客户端也有可能被自动（如果它们没及时与 MDS 通讯）、或手动驱逐（被系统管理员）。</p>
<p>客户端驱逐机制适用于所有类型的客户端，包括 FUSE 挂载客户端、内核挂载客户端、 nfs-ganesha 网关、以及其它使用 libcephfs 的进程。</p>
<section id="id1">
<h2>自动驱逐客户端<a class="headerlink" href="#id1" title="Permalink to this heading"></a></h2>
<p>在三种情况下，客户端会被自动驱逐：</p>
<ol class="arabic simple">
<li><p>在一个活跃的 MDS 守护进程上，如果一个客户端在限定的时间 <code class="docutils literal notranslate"><span class="pre">session_autoclose</span></code> （一个文件系统变量）秒数（默认
300 秒）内没有与 MDS 通讯，它就会被自动驱逐。</p></li>
<li><p>在一个活跃的 MDS 守护进程上，如果一个客户端在限定的时间 <code class="docutils literal notranslate"><span class="pre">mds_cap_revoke_eviction_timeout</span></code> （配置变量）秒数（默认
300 秒）内没有回应 cap 撤销消息。默认已禁用。</p></li>
<li><p>在 MDS 启动（包括故障恢复）期间，它要途径一个名为
<code class="docutils literal notranslate"><span class="pre">reconnect</span></code> 的状态，在这个状态下，它会等待所有客户端连接到这个新 MDS 守护进程。如果有客户端在限期（
<code class="docutils literal notranslate"><span class="pre">mds_reconnect_timeout</span></code> ，默认 45 秒 ）内未能连上，它们就会被驱逐。</p></li>
</ol>
<p>以上任何一种情况产生，就会向集群日志发送一条警告消息。</p>
</section>
<section id="id2">
<h2>手动驱逐客户端<a class="headerlink" href="#id2" title="Permalink to this heading"></a></h2>
<p>有时候，管理员可能得手动驱逐某个客户端。
比如说，一个客户端已死，管理员不想等它的会话超时；
或者有个客户端行为异常，管理员又无法登录客户端节点卸载它。</p>
<p>一般要先检查下客户端列表：</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">ceph</span> <span class="n">tell</span> <span class="n">mds</span><span class="mf">.0</span> <span class="n">client</span> <span class="n">ls</span>

<span class="p">[</span>
    <span class="p">{</span>
        <span class="s2">&quot;id&quot;</span><span class="p">:</span> <span class="mi">4305</span><span class="p">,</span>
        <span class="s2">&quot;num_leases&quot;</span><span class="p">:</span> <span class="mi">0</span><span class="p">,</span>
        <span class="s2">&quot;num_caps&quot;</span><span class="p">:</span> <span class="mi">3</span><span class="p">,</span>
        <span class="s2">&quot;state&quot;</span><span class="p">:</span> <span class="s2">&quot;open&quot;</span><span class="p">,</span>
        <span class="s2">&quot;replay_requests&quot;</span><span class="p">:</span> <span class="mi">0</span><span class="p">,</span>
        <span class="s2">&quot;completed_requests&quot;</span><span class="p">:</span> <span class="mi">0</span><span class="p">,</span>
        <span class="s2">&quot;reconnecting&quot;</span><span class="p">:</span> <span class="n">false</span><span class="p">,</span>
        <span class="s2">&quot;inst&quot;</span><span class="p">:</span> <span class="s2">&quot;client.4305 172.21.9.34:0/422650892&quot;</span><span class="p">,</span>
        <span class="s2">&quot;client_metadata&quot;</span><span class="p">:</span> <span class="p">{</span>
            <span class="s2">&quot;ceph_sha1&quot;</span><span class="p">:</span> <span class="s2">&quot;ae81e49d369875ac8b569ff3e3c456a31b8f3af5&quot;</span><span class="p">,</span>
            <span class="s2">&quot;ceph_version&quot;</span><span class="p">:</span> <span class="s2">&quot;ceph version 12.0.0-1934-gae81e49 (ae81e49d369875ac8b569ff3e3c456a31b8f3af5)&quot;</span><span class="p">,</span>
            <span class="s2">&quot;entity_id&quot;</span><span class="p">:</span> <span class="s2">&quot;0&quot;</span><span class="p">,</span>
            <span class="s2">&quot;hostname&quot;</span><span class="p">:</span> <span class="s2">&quot;senta04&quot;</span><span class="p">,</span>
            <span class="s2">&quot;mount_point&quot;</span><span class="p">:</span> <span class="s2">&quot;/tmp/tmpcMpF1b/mnt.0&quot;</span><span class="p">,</span>
            <span class="s2">&quot;pid&quot;</span><span class="p">:</span> <span class="s2">&quot;29377&quot;</span><span class="p">,</span>
            <span class="s2">&quot;root&quot;</span><span class="p">:</span> <span class="s2">&quot;/&quot;</span>
        <span class="p">}</span>
    <span class="p">}</span>
<span class="p">]</span>
</pre></div>
</div>
<p>找到想要驱逐的客户端后，就可以用它的唯一 ID 、或者其它属性来标识它：</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="c1"># 这些都可以</span>
<span class="n">ceph</span> <span class="n">tell</span> <span class="n">mds</span><span class="mf">.0</span> <span class="n">client</span> <span class="n">evict</span> <span class="nb">id</span><span class="o">=</span><span class="mi">4305</span>
<span class="n">ceph</span> <span class="n">tell</span> <span class="n">mds</span><span class="mf">.0</span> <span class="n">client</span> <span class="n">evict</span> <span class="n">client_metadata</span><span class="o">.=</span><span class="mi">4305</span>
</pre></div>
</div>
</section>
<section id="id3">
<h2>高级话题：从黑名单踢出客户端<a class="headerlink" href="#id3" title="Permalink to this heading"></a></h2>
<p>通常，进了黑名单的客户端就不能再重新连接服务器了：它必须被卸载、然后再重新挂载。</p>
<p>然而，在某些情况下，还是得允许被驱逐后、
又想重连的客户端。</p>
<p>CephFS 是用 RADOS OSD 黑名单来控制客户端驱逐的，
所以只要把它们从黑名单删除，就是允许这些 CephFS 客户端重连了：</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ ceph osd blocklist ls
listed 1 entries
127.0.0.1:0/3710147553 2018-03-19 11:32:24.716146
$ ceph osd blocklist rm 127.0.0.1:0/3710147553
un-blocklisting 127.0.0.1:0/3710147553
</pre></div>
</div>
<p>假如被拉黑的客户端对缓冲 IO 做了操作，而其它客户端访问了这些文件，这样做就可能会破坏数据完整性。而且也不能确定客户端功能会完全恢复——要想挽回一个完全健康的被驱逐客户端，最好的方法是把它卸载掉，然后再干净地挂载。</p>
<p>如果你想这样重连客户端，可以在 FUSE 客户端上设置 <code class="docutils literal notranslate"><span class="pre">client_reconnect_stale</span></code> 为 <code class="docutils literal notranslate"><span class="pre">true</span></code> ，
以此为客户端重连提速。</p>
</section>
<section id="id4">
<h2>高级话题：配置黑名单功能<a class="headerlink" href="#id4" title="Permalink to this heading"></a></h2>
<p>如果由于客户端主机慢或者网络不可靠，频繁遇到客户端驱逐，这些底层问题还解决不了，你可以让 MDS 别那么严格。</p>
<p>对于响应缓慢的客户端，可以简单地丢弃它们的 MDS 会话，却允许它们重新开启会话并且允许它们继续与 OSD 通讯。要打开这种模式，可以在 MDS 节点上把 <code class="docutils literal notranslate"><span class="pre">mds_session_blocklist_on_timeout</span></code> 设置为
false 。</p>
<p>对于手动驱逐时的类似情形，可以把 <code class="docutils literal notranslate"><span class="pre">mds_session_blocklist_on_evict</span></code> 设置为 false 。</p>
<p>注意，如果禁用了黑名单功能，那么驱逐客户端只会在收到了命令的 MDS 上生效。在一个多活 MDS 系统上，
你得向所有活跃的 MDS 守护进程发送驱逐命令。
黑名单功能启用时（默认如此），
只要把驱逐命令发给一个 MDS 就足够了，
因为黑名单会自动扩散到其余守护进程。</p>
</section>
<section id="osd">
<span id="background-blocklisting-and-osd-epoch-barrier"></span><h2>背景： 黑名单和 OSD 元图屏蔽<a class="headerlink" href="#osd" title="Permalink to this heading"></a></h2>
<p>一个客户端被加入黑名单后，有必要确保其它客户端和 MDS 守护进程能收到最新的 OSDMap （新增了这条黑名单的），以免有人再访问那些进了黑名单的客户端访问过的数据对象。</p>
<p>这是通过名为 “osdmap epoch barrier” 的内部机制保证的。</p>
<p>此屏蔽的目的是为了确保当我们分配出这些能力后，
别的客户端就有可能触碰同样的 RADOS 对象，
接受分配能力的客户端们必须有足够新的 OSD 图，
这样它们才不会与取消的操作（来自 ENOSPC ）或进了黑名单的客户端（来自驱逐过程）竞争。</p>
<p>更具体些，我们设置元图屏蔽的情形有：</p>
<blockquote>
<div><ul class="simple">
<li><p>客户端驱逐（此客户端被加入黑名单了、且其它客户端必须等到有加黑之后的元图出现才能触碰同一批对象）；</p></li>
<li><p>客户端正在处理 OSD 图的完整标识（此客户端可能会在快完整的元图中取消一些 OSD 操作，这样其它客户端都必须等着，直到元图完整或周期达到才能触碰同样的对象）；</p></li>
<li><p>MDS 启动，因为我们不会永久存储屏蔽图元，所以必须假设重启后总是需要最新的 OSD 图。</p></li>
</ul>
</div></blockquote>
<p>注意，为保持简洁性这是个全局值，其实我们可以做到按每索引节点维护此值，但我们没有这么做，因为：</p>
<blockquote>
<div><ul class="simple">
<li><p>它会复杂得多；</p></li>
<li><p>每索引节点需额外多占 4 字节的内存；</p></li>
<li><p>无论如何它都不会比大家一直都拥有最新的 OSD 图更有效。
而且，大多数情况下，大家都能轻松地越过这个屏障而不是等着它。</p></li>
<li><p>我们仅在极少数情形下使用这种屏蔽，所以每索引节点这样细粒度的实现带来的好处也很少见。</p></li>
</ul>
</div></blockquote>
<p>元图屏蔽随其它能力消息一起传递，
并且可指示消息接收器在看到这个 OSD 元图前、
别再向那些 OSD 发送 RADOS 操作。
主要是面向客户端（它们直接向文件写入数据）的，
也适用于 MDS ，因为像文件尺寸探测和文件删除这样的操作都是直接在 MDS 上进行的。</p>
</section>
</section>



<div id="support-the-ceph-foundation" class="admonition note">
  <p class="first admonition-title">Brought to you by the Ceph Foundation</p>
  <p class="last">The Ceph Documentation is a community resource funded and hosted by the non-profit <a href="https://ceph.io/en/foundation/">Ceph Foundation</a>. If you would like to support this and our other efforts, please consider <a href="https://ceph.io/en/foundation/join/">joining now</a>.</p>
</div>


           </div>
           
          </div>
          <footer><div class="rst-footer-buttons" role="navigation" aria-label="Footer">
        <a href="../snapshots/" class="btn btn-neutral float-left" title="CephFS 快照" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left" aria-hidden="true"></span> Previous</a>
        <a href="../scrub/" class="btn btn-neutral float-right" title="Ceph 文件系统洗刷" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right" aria-hidden="true"></span></a>
    </div>

  <hr/>

  <div role="contentinfo">
    <p>&#169; Copyright 2016, Ceph authors and contributors. Licensed under Creative Commons Attribution Share Alike 3.0 (CC-BY-SA-3.0).</p>
  </div>

   

</footer>
        </div>
      </div>

    </section>

  </div>
  

  <script type="text/javascript">
      jQuery(function () {
          SphinxRtdTheme.Navigation.enable(true);
      });
  </script>

  
  
    
   

</body>
</html>